{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![MLU Logo](../data/MLU_Logo.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <a name=\"0\">Machine Learning Accelerator - Tabular Data - Lecture 1</a>\n",
    "\n",
    "\n",
    "## Final Project \n",
    "\n",
    "In this notebook, we build a ML model to predict the __Time at Center__ field of our final project dataset.\n",
    "\n",
    "1. <a href=\"#1\">Read the dataset</a> (Given) \n",
    "2. <a href=\"#2\">Train a model</a> (Implement)\n",
    "    * <a href=\"#21\">Exploratory Data Analysis</a>\n",
    "    * <a href=\"#22\">Select features to build the model</a>\n",
    "    * <a href=\"#23\">Data processing</a>\n",
    "    * <a href=\"#24\">Model training</a>\n",
    "3. <a href=\"#3\">Make predictions on the test dataset</a> (Implement)\n",
    "4. <a href=\"#4\">Write the test predictions to a CSV file</a> (Given)\n",
    "\n",
    "__Austin Animal Center Dataset__:\n",
    "\n",
    "In this exercise, we are working with pet adoption data from __Austin Animal Center__. We have two datasets that cover intake and outcome of animals. Intake data is available from [here](https://data.austintexas.gov/Health-and-Community-Services/Austin-Animal-Center-Intakes/wter-evkm) and outcome is from [here](https://data.austintexas.gov/Health-and-Community-Services/Austin-Animal-Center-Outcomes/9t4d-g238). \n",
    "\n",
    "In order to work with a single table, we joined the intake and outcome tables using the \"Animal ID\" column and created a training.csv, test_features.csv and y_test.csv files. Similar to our review dataset, we didn't consider animals with multiple entries to the facility to keep it simple. If you want to see the original datasets, they are available under data/review folder: Austin_Animal_Center_Intakes.csv, Austin_Animal_Center_Outcomes.csv.\n",
    "\n",
    "__Dataset schema:__ \n",
    "- __Pet ID__ - Unique ID of pet\n",
    "- __Outcome Type__ - State of pet at the time of recording the outcome\n",
    "- __Sex upon Outcome__ - Sex of pet at outcome\n",
    "- __Name__ - Name of pet \n",
    "- __Found Location__ - Found location of pet before entered the center\n",
    "- __Intake Type__ - Circumstances bringing the pet to the center\n",
    "- __Intake Condition__ - Health condition of pet when entered the center\n",
    "- __Pet Type__ - Type of pet\n",
    "- __Sex upon Intake__ - Sex of pet when entered the center\n",
    "- __Breed__ - Breed of pet \n",
    "- __Color__ - Color of pet \n",
    "- __Age upon Intake Days__ - Age of pet when entered the center (days)\n",
    "- __Time at Center__ - Time at center (0 = less than 30 days; 1 = more than 30 days). This is the value to predict. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "%pip install -q -r ../requirements.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. <a name=\"1\">Read the datasets</a> (Given)\n",
    "(<a href=\"#0\">Go to top</a>)\n",
    "\n",
    "Let's read the datasets into dataframes, using Pandas."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The shape of the training dataset is: (71538, 13)\n",
      "The shape of the test dataset is: (23846, 12)\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "  \n",
    "training_data = pd.read_csv('../data/final_project/training.csv')\n",
    "test_data = pd.read_csv('../data/final_project/test_features.csv')\n",
    "\n",
    "print('The shape of the training dataset is:', training_data.shape)\n",
    "print('The shape of the test dataset is:', test_data.shape)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. <a name=\"2\">Train a model</a> (Implement)\n",
    "(<a href=\"#0\">Go to top</a>)\n",
    "\n",
    " * <a href=\"#21\">Exploratory Data Analysis</a>\n",
    " * <a href=\"#22\">Select features to build the model</a>\n",
    " * <a href=\"#23\">Data processing</a>\n",
    " * <a href=\"#24\">Model training</a>\n",
    "\n",
    "### 2.1 <a name=\"21\">Exploratory Data Analysis</a> \n",
    "(<a href=\"#2\">Go to Train a model</a>)\n",
    "\n",
    "We look at number of rows, columns and some simple statistics of the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Pet ID</th>\n",
       "      <th>Outcome Type</th>\n",
       "      <th>Sex upon Outcome</th>\n",
       "      <th>Name</th>\n",
       "      <th>Found Location</th>\n",
       "      <th>Intake Type</th>\n",
       "      <th>Intake Condition</th>\n",
       "      <th>Pet Type</th>\n",
       "      <th>Sex upon Intake</th>\n",
       "      <th>Breed</th>\n",
       "      <th>Color</th>\n",
       "      <th>Age upon Intake Days</th>\n",
       "      <th>Time at Center</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A745079</td>\n",
       "      <td>Transfer</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7920 Old Lockhart in Travis (TX)</td>\n",
       "      <td>Stray</td>\n",
       "      <td>Normal</td>\n",
       "      <td>Cat</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>Domestic Shorthair Mix</td>\n",
       "      <td>Blue</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>A801765</td>\n",
       "      <td>Transfer</td>\n",
       "      <td>Intact Female</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5006 Table Top in Austin (TX)</td>\n",
       "      <td>Stray</td>\n",
       "      <td>Normal</td>\n",
       "      <td>Cat</td>\n",
       "      <td>Intact Female</td>\n",
       "      <td>Domestic Shorthair</td>\n",
       "      <td>Brown Tabby/White</td>\n",
       "      <td>28</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>A667965</td>\n",
       "      <td>Transfer</td>\n",
       "      <td>Neutered Male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>14100 Thermal Dr in Austin (TX)</td>\n",
       "      <td>Stray</td>\n",
       "      <td>Normal</td>\n",
       "      <td>Dog</td>\n",
       "      <td>Neutered Male</td>\n",
       "      <td>Chihuahua Shorthair Mix</td>\n",
       "      <td>Brown/Tan</td>\n",
       "      <td>1825</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>A687551</td>\n",
       "      <td>Transfer</td>\n",
       "      <td>Intact Male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5811 Cedardale Dr in Austin (TX)</td>\n",
       "      <td>Stray</td>\n",
       "      <td>Normal</td>\n",
       "      <td>Cat</td>\n",
       "      <td>Intact Male</td>\n",
       "      <td>Domestic Shorthair Mix</td>\n",
       "      <td>Brown Tabby</td>\n",
       "      <td>28</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>A773004</td>\n",
       "      <td>Adoption</td>\n",
       "      <td>Neutered Male</td>\n",
       "      <td>*Boris</td>\n",
       "      <td>Highway 290 And Arterial A in Austin (TX)</td>\n",
       "      <td>Stray</td>\n",
       "      <td>Normal</td>\n",
       "      <td>Dog</td>\n",
       "      <td>Intact Male</td>\n",
       "      <td>Chihuahua Shorthair Mix</td>\n",
       "      <td>Tricolor/Cream</td>\n",
       "      <td>365</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Pet ID Outcome Type Sex upon Outcome    Name  \\\n",
       "0  A745079     Transfer          Unknown     NaN   \n",
       "1  A801765     Transfer    Intact Female     NaN   \n",
       "2  A667965     Transfer    Neutered Male     NaN   \n",
       "3  A687551     Transfer      Intact Male     NaN   \n",
       "4  A773004     Adoption    Neutered Male  *Boris   \n",
       "\n",
       "                              Found Location Intake Type Intake Condition  \\\n",
       "0           7920 Old Lockhart in Travis (TX)       Stray           Normal   \n",
       "1              5006 Table Top in Austin (TX)       Stray           Normal   \n",
       "2            14100 Thermal Dr in Austin (TX)       Stray           Normal   \n",
       "3           5811 Cedardale Dr in Austin (TX)       Stray           Normal   \n",
       "4  Highway 290 And Arterial A in Austin (TX)       Stray           Normal   \n",
       "\n",
       "  Pet Type Sex upon Intake                    Breed              Color  \\\n",
       "0      Cat         Unknown   Domestic Shorthair Mix               Blue   \n",
       "1      Cat   Intact Female       Domestic Shorthair  Brown Tabby/White   \n",
       "2      Dog   Neutered Male  Chihuahua Shorthair Mix          Brown/Tan   \n",
       "3      Cat     Intact Male   Domestic Shorthair Mix        Brown Tabby   \n",
       "4      Dog     Intact Male  Chihuahua Shorthair Mix     Tricolor/Cream   \n",
       "\n",
       "   Age upon Intake Days  Time at Center  \n",
       "0                     3               0  \n",
       "1                    28               0  \n",
       "2                  1825               0  \n",
       "3                    28               0  \n",
       "4                   365               0  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Implement here\n",
    "\n",
    "training_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Pet ID</th>\n",
       "      <th>Outcome Type</th>\n",
       "      <th>Sex upon Outcome</th>\n",
       "      <th>Name</th>\n",
       "      <th>Found Location</th>\n",
       "      <th>Intake Type</th>\n",
       "      <th>Intake Condition</th>\n",
       "      <th>Pet Type</th>\n",
       "      <th>Sex upon Intake</th>\n",
       "      <th>Breed</th>\n",
       "      <th>Color</th>\n",
       "      <th>Age upon Intake Days</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A782657</td>\n",
       "      <td>Adoption</td>\n",
       "      <td>Spayed Female</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1911 Dear Run Drive in Austin (TX)</td>\n",
       "      <td>Stray</td>\n",
       "      <td>Normal</td>\n",
       "      <td>Dog</td>\n",
       "      <td>Intact Female</td>\n",
       "      <td>Labrador Retriever Mix</td>\n",
       "      <td>Black</td>\n",
       "      <td>60</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>A804622</td>\n",
       "      <td>Adoption</td>\n",
       "      <td>Neutered Male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>702 Grand Canyon in Austin (TX)</td>\n",
       "      <td>Stray</td>\n",
       "      <td>Normal</td>\n",
       "      <td>Dog</td>\n",
       "      <td>Intact Male</td>\n",
       "      <td>Boxer/Anatol Shepherd</td>\n",
       "      <td>Brown/Tricolor</td>\n",
       "      <td>60</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>A786693</td>\n",
       "      <td>Return to Owner</td>\n",
       "      <td>Neutered Male</td>\n",
       "      <td>Zeus</td>\n",
       "      <td>Austin (TX)</td>\n",
       "      <td>Public Assist</td>\n",
       "      <td>Normal</td>\n",
       "      <td>Dog</td>\n",
       "      <td>Neutered Male</td>\n",
       "      <td>Australian Cattle Dog/Pit Bull</td>\n",
       "      <td>Black/White</td>\n",
       "      <td>3285</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>A693330</td>\n",
       "      <td>Adoption</td>\n",
       "      <td>Spayed Female</td>\n",
       "      <td>Hope</td>\n",
       "      <td>Levander Loop &amp; Airport Blvd in Austin (TX)</td>\n",
       "      <td>Stray</td>\n",
       "      <td>Normal</td>\n",
       "      <td>Dog</td>\n",
       "      <td>Intact Female</td>\n",
       "      <td>Miniature Poodle</td>\n",
       "      <td>Gray</td>\n",
       "      <td>1825</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>A812431</td>\n",
       "      <td>Adoption</td>\n",
       "      <td>Neutered Male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Austin (TX)</td>\n",
       "      <td>Owner Surrender</td>\n",
       "      <td>Injured</td>\n",
       "      <td>Cat</td>\n",
       "      <td>Intact Male</td>\n",
       "      <td>Domestic Shorthair</td>\n",
       "      <td>Blue/White</td>\n",
       "      <td>210</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Pet ID     Outcome Type Sex upon Outcome  Name  \\\n",
       "0  A782657         Adoption    Spayed Female   NaN   \n",
       "1  A804622         Adoption    Neutered Male   NaN   \n",
       "2  A786693  Return to Owner    Neutered Male  Zeus   \n",
       "3  A693330         Adoption    Spayed Female  Hope   \n",
       "4  A812431         Adoption    Neutered Male   NaN   \n",
       "\n",
       "                                Found Location      Intake Type  \\\n",
       "0           1911 Dear Run Drive in Austin (TX)            Stray   \n",
       "1              702 Grand Canyon in Austin (TX)            Stray   \n",
       "2                                  Austin (TX)    Public Assist   \n",
       "3  Levander Loop & Airport Blvd in Austin (TX)            Stray   \n",
       "4                                  Austin (TX)  Owner Surrender   \n",
       "\n",
       "  Intake Condition Pet Type Sex upon Intake                           Breed  \\\n",
       "0           Normal      Dog   Intact Female          Labrador Retriever Mix   \n",
       "1           Normal      Dog     Intact Male           Boxer/Anatol Shepherd   \n",
       "2           Normal      Dog   Neutered Male  Australian Cattle Dog/Pit Bull   \n",
       "3           Normal      Dog   Intact Female                Miniature Poodle   \n",
       "4          Injured      Cat     Intact Male              Domestic Shorthair   \n",
       "\n",
       "            Color  Age upon Intake Days  \n",
       "0           Black                    60  \n",
       "1  Brown/Tricolor                    60  \n",
       "2     Black/White                  3285  \n",
       "3            Gray                  1825  \n",
       "4      Blue/White                   210  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Implement here\n",
    "\n",
    "test_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 <a name=\"22\">Select features to build the model</a> \n",
    "(<a href=\"#2\">Go to Train a model</a>)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Implement here\n",
    "\n",
    "# numerical_features = ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 <a name=\"23\">Data Processing</a> \n",
    "(<a href=\"#2\">Go to Train a model</a>)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Implement here\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.4 <a name=\"24\">Model training</a> \n",
    "(<a href=\"#2\">Go to Train a model</a>)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Implement here\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. <a name=\"3\">Make predictions on the test dataset</a> (Implement)\n",
    "(<a href=\"#0\">Go to top</a>)\n",
    "\n",
    "Use the test set to make predictions with the trained model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Implement here\n",
    "\n",
    "# test_predictions = ..."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "sagemaker-distribution:Python",
   "language": "python",
   "name": "conda-env-sagemaker-distribution-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
